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Pile  report  ie  the  ninth  in  r series  puhlished  by  the  Blectronr- 
ics  Pv.eonnel  Seseerch  group.  3!he  first  seven  were  based  on  ship* 
board  observatipn  of  electronics  persosnel  nboard  ships  of  the  des~ 
troyer  class.  She  eighth*  ninth*  and.  tenth  reports,  describe  the 
results  of  collateral  research  which  was  done  concurrentlCT  with  this 
shipboard  observation.  Their  subject  matter  will  differ  from  that 
of  the  first  seven  reports;  conse<{uently  their  titles, are  listed  b»> 
low*  with  a brief  description  of  their  contents* 
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9*  h Methodological  Study  of  Electronios  Trouble  Shooting 
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flultiple^^teraative'^myol'ic  !rroubie^5oot ing  Test. 

A description  of  a new  type  of  test  format  designed 
for  measuring  some  aspects  of  trouble  shooting  sklllt 
and  a discussion  of  the  conception  of  trouble  shooting 
on  which  it  is  based. 
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i^mpie  Test*"^d  Ten  E^eri^e  Tests**Admini'8tere3[  t^¥leet 

A report  of  the  results  of  the  administration  of  two 
forms  of  the  Test*  its  progenitor  Job-^ssple 

test*  and  a battery  of  achievement  and  ability  oriented 
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This  report  is  one  of  two  oonoerning  a new  type  of  test  format 
which  was  a product  of  a methodhlogiohl  study' of  electronics  trouble 
shooting.*  A conception  of  trouble 'shooting  is  set  forth  as  it  is  re- 
lated to  problem  sdlving  in  general*  and  as  it  is  slesQ^lif led  in  elec- 
tronios  situations.  The  test  format  la  described  in  detail, ’and  the 
pertinent  aspects  of  its  subject  matter,  relation  to  other  trouble 
whooting  sltuatlone.  ai^  ^ate,  and  alteiuative  scoring  peiraaeters  are 
discUssedv 
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4 KigHQDOItOGIOAl  .BITODY  ’OT  aLBC^QglOS  IRCmS  SHOOTIS5  SKIlLl 
I.  RATIORAIB  PCJR  iiHD  DESCRIPTION  OP  THE  M^JITIPLE-ALTERDATIYE 
SBIBGLIO  TROUBLE  BHOOTINO  TEST 

1.  aiRODOOTION 

Thla  report  coaltalne  « deicrlptlon  of  tiie  Hultlple^Altenmtlve 
Syipatollo  Trouble  Shoot  lag  Test  and  the  rationale  underlying  It  a de» 
velopaent.  The  teet  la  a recult  of  a methodologlo4l  atudy  of  the 
probleaa  Inherent  in  the  aeamirettent  of  trouble  ahootlng  ahlll*  Coxh 
▼enticnal  paper-^and-pencil  teet  a may  be  ill-suited  to  thla  purpose. 
They  are  moat  useful  for  measuring  knowledge  presumably  related  to 
or  neoesaary  for  perforaanoe.  However,  their  format  generally  Is 
too  Inflexible  for  elloitlng  adequate  aemples  of  a performance,  Oon* 
ventional  Job-semple  teats,  on  the  other  hand,  evoke  performance  as-~ 
aumed  to  be  rapreaentatlve  of  that  on  the  Job  by  the  use  of  actual 
equismsent.  But  the  administrative  inconvenience  and  erpenso  of  equip- 
ment tosts  is  a ssrioua  practical  disadvantage.  So  is  the  fact  that 
they  are  inherently  difficult  to  score. 

The  MASTS  teat  wae  developed  ae  a new  kind  of  teet  format ' repre- 
senting the  ooaiplexitiea  of  the  trouble  shooting  situation  with  suf- 
ficient realism  to  evoke  trouble  shooting  performance,  while  pOsseae- 
Ing  some  of  the  administrative  ooaveaience  of  a paper-aM-p^oll . 
formii^.  It  uniquely  oomblnes  these  charaoteylatloat 

1«  It  forces  the  eul^eot  to  determine  hie  own  path 
. , to  the  solution 'Of  a trouble  sJsooting  problesi,  - • 

No  standard  list  of  altwnatlyes  is  presented, 
for  him  tO'foUo*.  ■ 


2,  The  diecrialnation  of  '^ytsulno”  or  "in  tolsrance/out 
of  tolerance,"  ia  not  made  for  the  aubject  with 
reapect  to-  the  problem  information.  He  hae  to  make 
theae  diacriminationa  for  himaelf, 

3*  !The  problem  information  waa  en^iricalljr  determined. 
There  are  no  theoretical  Talviea,  which  ao  often 
differ  from  readinga  obtained  from  the  actual  equip- 
ment. 

4,  An  extremely  large  eaount  of  information  ia  available 
for  each  trouble  ahooting  problem.  The  aubject  haa 
to  find  cuss  to  aclution  of  a problem  in  a pool  of 
approximately  five  hundred  different  readings* 

poeaible  to  record  a detailed  protocol  of  the 
subject' a behavior  in  the  testt  the  sequence  of  his 
reeponaes,  the  points  in  the  cirouit  at  which  ho 
Bonplea  information,  the  number  of  times  he  refers  to 
the  sohematie,  and  the  time  of  occurrenoe  of  theoe  re- 
apouaea  can  be  racorded  on  a standard  form  by  means  of 
a coding  system.  These  response  reoorda  provide  the 
basis  for  studying  differences  in  technique  between 
> - V irixcooosfuL  aiid-UxuiuopeaBfia  subject 


A later  report  will  contain  the  resulta  of  administering  this  teat 
a Job-sample  test,  and  a battery  of  reference  teats  to  a sample  of  ETa 
from  ships  undergoing  repairs  at  the  Long  Beach  Haval  Shipyard. 


II.  THE  BASIS  OS’  TSS  H4STS  TEST 


This  section  contains  a diBCuaeioa  of  the  Job  analysis  done  pre- 
liminary to  construction  of  the  teat.  Out  of  this  ar.alysie  came  the 
conception  of  electronic  trouble  shooting  from  which  the  test  format 
evolved. 


i],  *.  <iC‘'  4 
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h'  Shoot  lBg;^and  SLeotronio  Circuits 

Observation  cf  SSs  parforming  correct ive  maintenance  aboard  dea- 
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troyera  operating  off,  Pacific 'Coast  .study  of  trouble  shooting  is 
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the  UaiVertltF  lfthor«itory«  acid  consideration  of  reports  fron  other 
investigators  provided  a haokground  of  experience  fundamental  to  de= 
velopment  of  the  test. 

She  situation  in  ^ioh  trouble  shooting  occurs  has  t%ro  prominent 

* ^ 

elements*  an  individual  or  individuals*  and  a malfunctioning  sy'stem* 
It  is  the  task  of  the  iadividusl  (e)  to  fix  the  esretem.  i-ttes^te  to 
find  what  le  causing  the  nalfunotloning  sure  called  trouble  shooting 
behavior*  which  is  viewed  here  as  a spec  if  io  kind  of  problesHsolvisg 
aetivitjr. 

. 4 

Certain  elements  of  the  interaction  between  the  individual  and 
the  BTStem  are  common  requirements  of  eucoessful  trouble  shoot ingt 
the  individual  seust  (i)  have  some  knowledge  of  how  the  system  funo<* 


sally*  \Zf  obtain  inforaati^  about  the  ourfeni  state  of 
the  system*  (3)  relate  the  information  he  gets  to  his  conesption  of 
the  non^  system*  his  past  nxparienoe  with  malfunctions  of  this  or 
slmilir  systems,  and  his  theoretical  knowle^e  of  functional  rela* 
tionshlps  embodied  in  the  system*  and  (4)  forsulate  and  test  hy- 
potheses as  to  the  moat  probable  cause(s)  of  the  malfunction, 

She  characteristics  of  the  system  «nd  the  complexity  of  the 
cause(s)  of  the  malfunction  are  principal  determinants  of  the  dif- 
fioulty  of  the  trouble  snooting  task,  electronic  circuits  possess 
some  oharaeteristics  peouliar  to  them,  Host  of  these  stem  from  the 
eli^le  fact  that  the  electraa  is  IryieibXe,  Sxoept  at  a few  Isolated 
^ia  in  oi^u^  idwir^  il^oi^tion, carried  by  it  ;ie  tssuselated 


She  ay  item  may  be  an  ^eot  rondo  circuit*  an  internal  ooaibuetlon 
eagiiMf  a eoeial  institution  ojr  b businese  procedure. 


intc  eensory-  terms  aa  output  devicet  the  stream  of  electrons  flovis^ 

In  the  equlpmenti  and  the  complex  functional  interaction  of  its  parts, 
are  ahatract  concepts.  A wire  with  a potential  of  10.000  volts  looks 
Juet  like  a wire  with  no  voltage  at  all. 

Shis  attribute  of  functional  invisi'bilit7  places  certain  demands 
on  the  trouble  shooter  in  eleotronios.  fhese*  while  probably  not  unique, 
are  believed  to  have  a prominent  place  in  the  constellation  of  aubtasks 
which  compose  this  trouble  shooting.  It  requires  him  to  know  and  to 
interrelate  two  different  representations  of  a circuit  - a theoretical 
conception  made  up  of  abstract  concepts  and  based  on  a echemtioj  and 
its  actual  physical  tranalation  in  the  form  of  the  extremely  complicated 
arrangement  of  leads,  tubes,  capacitors*  resistors,  etc,,  elec- 

tronio  equipmeut. 

The  t^rouble  shooter  is  required  to  shift  back  and  forth  from  one 
representation  to  the  other;  he  is  guided  by  abstract  concepts  on  the 
symbolic  level  while  using  tools  and  test  instruments  at  the  physical 
level.  As  a result,  he  tends  to  build  up  for  himself  a functional  vis~ 
ualisation  of  tho  circuit  emboilsd  in  a particular  pleca  of  equipment 
with  which  he  has  contact  a functional  cone  apt  ioaalizat  ion  which  is 
a product  of  the  shifting  from  one  representation  of  the  circuit  to 


ft 


G. 


}7ot  the  least  of  the  many  troublesome  differences  between  these 
two  representaUonsare  the  facts  that  the  Icyout  of  coFiponentB  In  th«> 
equi^ffisent  never  corresponds  to  the  spatial  arrai^ement  of  their  syi^bols 
on  tha  soheasatiot  and-ths  differences  between  coasted  values  at  joints 
in  a theoretical  circuit  usd  the  actual  values  at  correqpondii^  ]^int8 
in  equipment  bMed  ou  tlMsi’^elrculi  W ooh8ldex«.hleti  Inter- 

actions between  coiqponent  tolerances,  alight  line  voltage  fluctuations, 
slightly  weak  oomposents,  and  the  effects  of  test  Instrusi^ita-ou 
values* 


tho  other  vhlle  trouble  ehootlag.. 


Analyele  of  the  Trouble  Shoot ia«  Task  in  Electroglcs 

•W  mmomimmKmmmmmm  mmm  tmmuum  mmmmmmmamem  rnrnrn^immmmSmi  momuh*  aeuv  ..  m si  n 1 1 vmrmvmmmmmm 

On  the  premise  that  trouble  shooting  in  electronlos  merely  is  a 
specific  example  of  a general  class  of  trouble  shooting  behavior,  its 
analysis  can  follow  the  frame  of  reference  establlehed  by  the  four 
requirements  which  were  listed' for  trouble  shooting. 

1.  Knowing  the  Kormal  System.  The  is  or  other  technician 
brings  to  the  trouble  shooting  eituation  an  aecnaulation  of  knowledge 

. 3 

about  electronic  eystens  which  he  has  picked  up  from  various  sources. 
In  addition,  he  is  given  a certain  amount  of  information  in  the  fora 
of  schematics,  photographs,  and  other  reference  material  .^mpanylng 
the  equipment,  such  as  tables  of  normal  plate  voltage  readings  and 
‘ the  values  of  circuit  ooi^ononts.  " ' 

2,  Petting  Information  About  the  Present  State  of  the 
System.  There  are  two  general  categories  of  this  information*  cues 
immediately  available  to  the  unaided  senses  and  cues  which  must  be 
transformed  into  sensory  stimuli  by  an  intermediate  instrument, 
amplea  of  tha  former  are  front  panel  Indications,  output  ^mptoms. 


J 

It  should  be  pointed  out  here  that  some  ITs  apparently  are  able 

♦.ft  4 VTA 


with  \idiich  they  have  had  lo:^  experience,  on  the  basis  of  probability 
of  recurrence  of  a particular  ^Toptom-causa  relationship  which  has 
occurred  repeatedly  in  the  equipmont’s  past  history.  The  extreme  fora 
of  this  is  not  regarded  as  trouble  eiiooting,  since  it^  ^s  not  in^lve 
probl^  solving.  The  ST  merely  repeats  a sequence  of  operations  which 
have  ih:  the  past,  under  similar  circunustaaces,  put  the  equipment  back 
en4he  air.  An  example  cf  this  type  of  corrective  nAintenance  was 
noted  on  a destroyer,  where  a leah  In  the  oyeyhead  ^tnr  to^ 

drip  into  a transmitter  at  rather  frequent  intervals,  thxtil  the  deck 
abqve  was  fixed,  the  STs  isuaediately  replaced  the  same  parts  in  the 
transmitter  every  time  it  was  reported  down,  without  boti»ring 
to  check  syBgptomis. 


cold  tu1)6S(  leaking  capacitor  canct' Bcoking  resictore*  etc.  Sxasiples' 
of  the  latter  are  vbltage  readings,  waveforma,  ring  time,  etc,  The  de- 
vices which  perfoimi  the  transformation  are  test  instruments,  and  use  of 
such  instruments  is  an  important  part  of  the  trouble  shooting  task. 

To  get  information  about  the  present  condition  of  the  equipment, 
then,  the  technician  may  perform  a careful  preliminary  search  for  cues 
immediately  available  to  his  senses,  or  he  may  resort  to  auxiliary  de- 

■ .»  V 

vices  to  sample  information  at  various  points  in  the  circuit.  Both  of 
these  procedures  involve  three  stepst  (a)  choice  of  the  information^ 
sampling  device  or  process,  (b)  selection  of  points  in  the  circuit  at  , 
which  to  sample  Information,  and  (c)  correct  use  of  the  chosen  informa- 
tion-sampling device  or  process, 

'i  * ■ * 

Information  About  the  Present  Condition  of  the 

System  to  a Oonoaptlon  of  Its  Normal  Oondit ion.  The  trouble  shoot has 

1 * * • 

to  do  something  with' the  information  he  gets  by  his  sampling  processes. 

• ■ * i * 

As  he  collects  it,  he  necessarily  attes^ts  to  classify  it  by  counting 
what  the  functional  values  at  particular  points  theoretically  should 
be,  or  by  comparing  the  readings  he  had  taken  with  a list  of  values  sup- 
plied In  supplementary  reference  material,  or  by  collaring  them  with 
standards  based  on  his  e^erience  with  the  particular  equipment.  All 
of  these  processes  ax9  based  oni  (d)  the  discrimination  of  usual  or  un- 
usual for  each  bit  of  information  collected  from  the  equipment  by  a 
sampling  process,  • . - • , 

Ofaecking  Possible  ^d  Probable  Causes  of  the  Half i^ct ion. 
So  far*-  the  processes  described  are  relatively  straightforward 
cations  of  ohservational  techniques  to  a mal^xjnctioning  systeffl. 


theaselvest  tbogr  night  result  in  the  aocunulatlon  of  & large  sazqple 
of  prohlen  laforaation.  Bovever.  unless  this  Infonaatlon  were  Inte^ 
grated  Into  a cause-effeot  frame  of  reference*  it  would  he  insuffic- 
ient for  solving  the  trouhle-shooting  problem* 

Another  level  of  the  behavior  begins  some  time  before  or  during 
the  application  of  these  techniques*  and  goes  on  concurrently  with 

- j < 

them.  At  sons  stage  in  the  task*  the  trouble  shooter  begins  to 
forsnilate  hypotheses  as  to  the  possible  and  probable  causes  of  the 
malfunction.  She  basis  for  this  formulation  may  be  the  relation- 
ships he  draws  betwesn  the  information  he  collects  and  his  functional 

-•'■4 " ■ 

conceptual  i sat  ion  of  the  equipment* 

To  be  an  efficient  or  even  a successful  trouble  shooter*  the 
technician  must  avoid  either  the  extrsms  cf  getting  all  the  infers&= 
tion  potentially  available  or  of  rebuilding  an  electronic  system 
poaqponent  by  component,  fie  must  achieve  some  eoonomioal  balance 
between  information  sasqpling,  and  the  risk  that  hit  informed  guesses 
as  to  causes  of  patterns  of  unusual  information  will  be  wrong.  Qen- 
erally,  he  achieves < this  balance  selecting  the  points  at  which  he 
samples  circuit  information  in  such  a way  that  hs  can  eliminate  por- 
tions of  the  equipment  as  possible  loci  of  the  trouble  each  time  he 

fiowever*  a trouble  shooter  does  not  necessarily  have  to  sample 
Information  first.  It  is  possible  for  him  to  make  random  guesses 
abpjdt  the  cause , of  a failure  in  a syetea  end  to  be  correct  some  times. 
The  more  complicated  the  system*  the  greater  the  number  of  possible 

of  fnd  thf,  e ,i^,|hle  ge#e»ing  apt  te  be  successful* 

Conversely*  "the  more  a troubie  shooter  knowe  abof:.t  the  present  con^. 

:.41tlpn  of  the  mystem  iU|X^^a%ioa  t4  norpal  pondition,  the  iK>re 
his  informed  guesses  apt  to  be  correct*  provided  he  can  deduce  cause- 
effect;  reiAtienships. . for-  t^e 4i«Prepanp lee  -he  notea.'' "£ 


doee  th6  BampXing.  This  arrowing  of  the  field,  of  po,  a Bible  couseB  per-  = 
mite  the  tschnioieji  to  check  a smaller  number  of  alternative  probable 
o^soB,  one  or  more  of  which  will  be  the  reed  source(s)  of  the  malfunc- 
tion. 

IThie  elimination  of  posaible  and  probable  causes  Is  conceived  of 
as  a product  of  the  first  three  req]airement8  discussed  above,  and  this 
fourth  requirement,  which  includes  these  stepst  (e)  integration  of  the 
information  as  it  is  collected  into  a cause-effect  frame  of  reference, 
and  (f)  testing  hypotheses  as  to  probable  causes  of  the  trouble.  . 

• • Iroli bat ions  for  Best  Construction 

This  discussion  of  trouble  shooting  is  intended  to  suggest  certain 
conclusions  about  the  nature  of  trouble  shooting  beia-vior  which  have 
served  ag  the  Underlying  assumptions  of  the  HAST5  Test,  It  is  net  im- 
plied the  anilysla  represent-  the  way  all  ST-e  trouble  shoot,  for  at 
least  some- technicians  do  net  follow  a system,  and  cannot  verbalise  the 
^ .'Structuring  principles  cf  their  behavior  in  this  situation. 

•The  analysis  does  show,  however,  that  the  trouble  Shooting  task  is 
composed  of  heterogeneous  subtasks,  each  requiring  certain  aotlvitles 
which  are  determinants  of  the  success  or  failure  of  the  trouble  shooting 
effort*.  ... 


the  problem  for  himself.  He  starts  with  a certain  number  of  *glv(ms*' 
in  the  situation:  output  symptoms,  front  panel  indications,  operator^ 
repoTt&i  supplement^r'y  reference  material',  and  his  own  experience. 


tween  this  istaft  and  the  ehd  cf  the  pibblea  Ties  a solution  ^ut« 
he  detemines.  It  may'  have  nmeTous  byt^tys.  or  it  may  proceed  by 'thb 


shortest  possihle  path  from  start  to  suocotsful  solution  of  the  proV 
Ism.  She  point  to  reitsrate  is  that  this  path  is  a r«:^ult  of  the 
interaction  hetveen  the  situation  and  the  indiTidohl*  Ho  two  teehp* 
mioians  solve  the  same  prchlem  in  exactly  the  same  way.  Share  ars 
no  flxnd  alleys  in  the**maie*  they  traverse.  !^h  individual  selects 

t 

...  - . • -.-i  . ’ f 

his  own  test  points  In  the  circuit  asii  ii  faced  hy  his  own  choice 
points,  each  of  which  is  a resultant  of  some  of  his  behavior  up  to 
that  ttoae, 

^ Shis  concept ioh  rejects  the  view  that  the  trouble  shooter’s  suc> 
c^esive  responses  are  rigidly  determined  by  his  preceding  respoivses* 
inalysls  of  detailed  resichse'  records  has  shown  that  even  the  suc- 
cessful trouble  shooter  icskes  both  fruitful  and  unfruitful  moves* 

He  may  indulge  in  repetitive  hirisacviorrmBihl^'t^^^  same  measurement 
^ over  and  over  at  a test  point.  Or,  he  may  skip  about  in  the  circuit 
with  no  apx>arent  relationship  among  his  responses.  fhe  idea  that 
the  first  response  determines  what  the  second  response  will  be,  and 
that  one  and  two  determine  the  nature  of  the  third,  presents  too  rigid 
a picture  of  the  process. 

. It  may  be  t>iat  this  idea  would  tend  to  be  more  and  more  ap- 
plicable as  the  efficiency  of  the  trouble  ahootere  increased,  but 
a test  which  forces  all  subjaeta  to  choose  the  same  alternatives  at 
the  same  choice  pointe  would  omit  an  ia^rtant  source  of  variance* 

Jl  technician’s  ftultfxil  responses  move  him^  dlceer  to  a solution,  and 


in  that  sense  are  successive]^  depen^nt'^  but  this  relationehlp  holds 
03^  __^ong  . certain  re^pnaee  c;C^eh.a,opafaS?f  ; 


i3onvsnticnal  psgjsr-anA-penoll  test  forsate  are  ini.de<joate  for  prs- 

'■i>  • . • ■ ■ 

eenting  realistic  trouble  ehootlug  problems  to  a subject.  I!he7  tend  to 
rigidly  structure  the  technician's  path  from  beginning  to  finish  by  sup» 
plying  a very  limited  number  of  standard  alternatives!  choice  points* 
and  samples  of  problem  Information!  and  they  give  away  Information  by 
listing  alternatives  and  crucial  cues. 

The  Ideal  format  for  a measure  of  trouble  shooting  behavior  would 
start  the  subject  with  a standard  minimum  of  information  about  a problem 
and  then  force  each  subject  to  structure  his  own  presolution  behavior. 

It  would  require  the  subject  to  use  the  procissses  which  have  been  dis- 
cussed In  the  analysis  of  electronics  trouble  shoot lng»  and  which  are 

summarised  heret 
« 

1,  Knowing  the  Normal  System 

2,  Getting  Informat  Ion  About  the  Present  State  of  the^  System 

a.  Choice  of  the  Informat Ion- sampling  device  or 
proosss. 

b.  Selection  of  points  In  the  circuit  at  which  to 
sample  information, 

c.  Correct  use  of  the  chosen  Informations sampling 
device  or  process, 

3,  Belating  the  Information  About  the  Present  Condition  of 
the  System  to  a bonceptlon  of  It s NormaV  Condition"** 

d.  The  discrimination  of  usual  or  unusvial  for  each 
bit  of  information  collected  from  the  equipment 
by  a sampling  process. 

Checking  Possible  and  Probable  Causes  of  the  Malfunction 

e.  The  Integration  of  the  Information  as  It  Is  col- 
lected into  a cause-effect  frame  of  reference. 

f.  The  testing  of  hypotheses  as  to  probable  causes, 

of  the  trouble.  . ' ; 

This  conception  required  a hew~ kind  of  tost  format."^  After  a con- 

aiderable  amount  of  preliminary  anaiysis,  the  structure  of  the  Multiple- 


Altematl¥d  S^^Xie  ’Xrouols  SrAotixig  Test  was  developed^  Th^  te^ 

.*.»*.  •• 

nalnder  of  the  paper  will  l>e  devoted  to  a description  .of  its  fonoat 
and  a discussion  of  its  charaoteristlos. 


111.  21SGEIFTX0S  0?  TES  ifASTS  TSST 

•Hhe  isQportant  ohsraoteristios  of  the  KdSTS  Test  will  1>s  desorihed 
in  this  section.  The  present  j^sieal  fom  of  the  test  is  regarded 
merely  as  one  possihility  for  the  enhodiaent  of  its  principles.  The 
e:cploded  drawings  in  Tigs.  1 and  2.  and  the  photographs  (Plate  l) 
acccaqpanying  the  text  give  the  reader  structural  details  of  the  ctu>- 
rent  fora. 


K The  Test  Toraat 

general  req.uireaents  listed  in  the  precedi^  section  isqply 
either  a ^oh-sassple  test  or  some  arrangessent  which  would  replace 
both  the  eleotronio  equipment  and  the  test  instruments  in  the  trouble 
shooting  situation,  while  reproducixig  a sufficient  number  of  the 
subtasks  in  it  to  evoke  representative  trouble  shooting  behavior.  To 
accomplish  this,  the  test  should  provide  the  subject  with  a pool  of 
information  about  each  problem  which  ho  can  sasple  according  to  his 
own  Inclinationa.  The  test  also  should  possess  face  v^ldity  and  . 


Interest;  it  should  make  the  subject  feel  that  he  is  trouble  shooting, 
. T^^h©  apparatus  which  built  is  shown  in  Tig.  1,  The  following 
discvLsslo;i  refers,  to  the  awbered  parts  of  this  drawing.  This  is  the 


labo^  the  test*  d prodiwtlon  s|odel  could  bjc  reduced  in 

slse  €Oid  weight  and  sjm&e  other  refinements  could  be  introduced. 


She  maBonlte  hackhoard  (1)  acts  aa  a atipport  in  the  current  model. 

It  is  approximately  4x5  feet.  Along  the  "bottom  edge,  grooved  atrip 
(A)  serves  to  hold  the  ends  of  the  five  panels  in  place. 

She  pool  of  pro"blem  Information  is  typed  on  the  sheet  (2).  Shore 
is  a different  sheet  for  each  problem,  for  the  sake  of  convenience  in 
typing  and  handling!  this  sheet  has  been  made  in  five  strips « each  fib- 
ting  \inder  one  panel.  She  information  is  typed  on  the  strips  in  circlesi 
which  were  drawn  by  running  a pencil  around  the  holes  in  the  super- 
irnpesed  panels  (3)« 

She  problem  information  presented  to  the  subject  is  of  three 
classes  t readings  made  at  test  points  in  the  circuits  which  were 
chosen  as  contexts  for  the  .rouble  shooting  problems,  information  which 
would  be  obtained  from  manipulating  front  panel  controls  and  screwdriver 
adjustmente  on  actual  equipment  • and  the  answers  to  the  problems.  These 
three  claeses  of  information  will  be  described  more  fully  later. 

fhe  five  panels  (3)  are  masonite,  fheir  bottom  ends  fit  under  the 
strip  (a)  on  (l),  5he  top  ends  of  the  panels  are  clamped  to  (l)  by  air- 
craft rigger^ B clamps  (B).  In  these  panels  are  drilled  3/4”  holee.  On 
each  of  four  panels  there  are  five  coltimns  and  25  rows  of  these  holes. 

On  the  fifth  panel,  there  are  seven  colximne  of  holee,  one  coltusn  for 
’'vary*'  information  and  six  for  the  **answer”  section  of  the  test.  Saoh 
hole  in  the  panels  is  ’’stoppered*  with  a cork  (I'lg.  2),  In  the  first 
four  panels,  all  the  corke  in  a particular  one  of  the  five  columns  per 
panel  are  painted  a dittinctive  color.  She  small  strijps  (0)  ubntain  the 
labels  for  tbe  cdltmns  of  l^re^  tftrip  ^©‘^satie  flvie  laibbla, 

one  for  each  of  the^f  ive"  columns  bn  a panbl;  These  lab el^'  c 


FIG.  1.  DIAGRAM  OF  MASTS  TEST  FORMAT. 


to  the  five  kinds  of  infonaatlon  available  to  the  subject  at  each 
test  point  in  the  circuit.  Details  of  the  etrips  are  shown  in  the 
esqpsnded  view  in  Fi^*  2.  Each  row  of  cozies  on  the  four  similar 
panels  is  numbered  conseoutiveljr  from  1 to  100*  the  nun^ers  cor- 
responding to  possible  test  points  in  the  circuit. 

The  cozies  in  the  large  panel  are  labeled  on  tep  with  the  letter 
and  number  Identifying  a conqponentt  including  major  leads*  in  the 
circuit  being  used  as  the  context  for  a problem.  The  first  column 
In  this  panel  is  for  "Vary"  information  - that  which  would  be  ob- 
tained fremi  varying  front  panel  controls  and  screwdriver  adjustments. 
The  other  six  columns  in.  the  panel  are  for  the  answers  to  the  prob- 
lems. These  columns  are  labeled  "Substitute  Hew  Cos^onent. " For 
any  particular  problem,  one  of  the  corlcs  in  the  answer  section  cor- 
responds to  the  component  in  the  circuit  which  caused  the  malfunction. 
Under  ic  is  printed  the  phrase  "Gear  Normal,"  The  spaces  under  the 
other  corks  in  the  section  are  left  blank. 

To  facilitate  locating  them,  the  corks  in  the  answer  section  are 
grouped  in  rows  labeled  with  the  names  of  the  stages  in  the  circuit. 
The  strip  (D)  is  fastened  along  the  edge  of  the  section  and  carries 
these  stage  labels.  It  and  the  other  labels  (C)  can  be  changed  when 
different  circuits  or  different  test  Instruments  are  represented. 

All  of  the  cozks  can  be  removed  and  replaced  readily.  They 
eerve  to  cono^al  problem  information;  the  response  of  removing  and 
replacing  a cork  corresponds  to  using  test  instruments,  varying  front 
panel  oozztrole,  making  sorewdriver  adjuetmezzte,  or  substituting  new 
oojqpon^ta  in  the  actual  eqxiipment.  Numbers  (4)  and  (3)  nre  the 
switch, aad  r^  warning  reap«ativ^.  ^ !Kiay  are  the 
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ohisf  feat'urfs  a tlae  delay  eircuit*  • 

3.  !Iha  Subject  Matter  of  the  Teat 

As  a gymholic. representation  of  a trouble  shooting  situationt  the 
MASTS  Tost  Is  baaed  on  actual  equipment.  Two  circuits  were  chosen*  both 
arailable  in  the  Phllco  Gaining  Set  (Plate  II) , a superheterodyne  radio 
receiver  and  a radar  sweep  generator  circuit.  The  Phllco  equipment  la 
relatively  portable  and  eobodlea  general  principles  which  are  found  in 
more  complicated  electronic  ayeteme.  However*  other  circuits  could  be 
used  for  the  test.  The  accesalblllty  and  relative  simplicity  of  the 
Phllco  chassis  facilitated  the  x^eliminary  work  which  was  necessary  to 
establish  the  information  pools  for  each  problem*  and  to  study  the  char> 
acteristiCB  of  trouble  shooting  problems. 

One  of  the  features  of  the  test  was' to  be  its  realism.  The  prob~ 
lem  information  wae  an  important  consideration*  since  it  either  could 
be  ’’fakad”  or  it  could  be  obtained  by  putting  faulty  components  in  the 
equipment  and  using  test  instromants  at  test  points  to  get  empirical 
measures  of  the  effects  of  the  faults  on  the  rest  of  the  ciroult*  An 
appraisal  of  poBsibilitiss  indioated  that  estimating  or  coaputlng  the 
values  for  the  test  points  would  be  a.A^stionable  procedure.  It  might 
have  resulted  in  lists  of  theoretical  values*  but  these  would  have  had 
important  deviations  from  those  a technician  might  obtain  for  himself 
whan  using  test  instruments. 

Therefore*  the  decision  was  made  to  get  the  problem  information  ly 
empirical  means.  Correct  echematics  of  the.  two  circuits  were  drawn. 

Test  points  were  selected  to  represent  points  in  all  the  circuit  stages 

5 — . 

This  is  part  of.  e system  which  vss  devised  tc  rsproducs  some  of 
the  effect  the  effort  Involved  in  r«soval  and  replacement  of  a compon** 
ent  in  actual  gear  has  on  the  technician’s  willingness  to  do  so  until, 
he  feels  it  is.. necessary..  This  task  acts  as  a deterrent  to  excessive 
removals.  In  the  MASTS  Test*  the  subject  is  given  a certain  amp\int  of 
time  to  solve  ee^h  problem.  To  prevent  him  from  doing  so  by  merely 
lifting  all  the  corks  in  the  answer  section  of  the  large  panel*  he.  is 
required  to  perform  the  .following  sequence  of  operations  before  ha  re- 
moves a cork  thefet  push  toggle  switch  (4)  which  turns  on  the  red  warn- 
ing light  and  activates  a timer  (not  shown  in  the  drawing);  wait  thirty 
seconds  for  the  timer  to  t\im  the  red  light  off:  when  it  goes  off  he 
can  remove  the  coack  but J8^t  rspe^  sequence  before  another  removal, 
in  that  section.  ' ■ ’ 


at  which  a technician  mi^ht  want  to  san^le  information.  There  were 
approximately  90  such  teet  points  in  aaoh  circiiit.  Hext»  the  kinds 
of  readings  to  take  were  chosen,  keeping  In  mind  the  test  instruments 
ETs  commonly  use.  For  the  radio  receiver  these  were  DC  volts,  AO 
volts,  ohms,  signal  injection  and  taps.  Tor  the  radar  circuit  they 
were  waveforms,  SO  volts,  AC  volts,  and  ohms.  Saush  of  these  kinds  of 
readings  were  made  at  each  test  point  in  the  appropriate  circuit, 
which  resulted  in  from  3^  to  425  readings  per  prohlem. 

This  procedure  resulted  in  empirical  information  which  was  real- 
istic, containing  as  it  did  all  the  minor  fluctuations  due  to  the 
characteristics  of  the  test  instruments  and  the  equipment. 

Prior  to  this  work,  it  was  necessary  to  select  the  trouble  shoot- 
ing problems.  The  Initial  selection  was  made  to  sample  each  impor- 
tant  stage  in  each  circuit,  end  each  common  type  of  failure  of  each 
type  of  cos^onent.  As  the  readings  were  taken,  some  problems  which  . 
had  been  chosen  proved  to  be  unsuitable.  Some  faulty  co^onents  made 
toe  little  difference  in  the  operation  of  the  equipment.  Certain 
problems  had  key  sensory  cues  which  could  not  be  adequately  repre- 
sented symbolically.  Interacting  faults  had  to  be  omitted  because  of 
the  difficulties  inherent  in  representation  of  their  syffiptoms  and 
cues.  The  final  election  includes  twelve  radio  receiver  problems 
and  twelve  radar  problems.  These  were  divided  into  two  equal  groups, 
to  provide  two  sets  of  problems,  either  of  which  could  be  used  in 
the  MASTS  Test.  The  two  sets  were  matched  pn  the  basis  of  their 


; Another  example  of  the  divergence  between  the  theoretical  coz^> 
ception  of  a circuit  and  its  physical  translation. 


•leetronic  chareetisristlps,' 

M present  • then,  the  Test  has  two  i&ore  or.  lees  e<j.uivaXent 

forms  so  far  as  subject  matter  is  conoerned.  In  each  there  are  six  re» 
ceiver  problems  and  six  radar  problems.  All  of  the  information  for  each 
of  these  problems,  except  initial  sys^toms  and  supplementary  reference 
material  about  the  normal  condition  of  the  circuit,  is  on  the  problem 
sheets.  This  supplementary  reference  material,  consisting  of  a schematic 
with  the  test  point  nvmhers,  instructions. to  the  subject,  normal  plate 
▼oltBgee,  and  values  of  major  components  in  the  circuit s,  is  in  a sepa- 
rate  booklet* 

2.*  ^o^lea  ♦laf  onaation 

The  procedures  used  for  getting  the  problem  information  deserve 
sane  coninen-t,  since  it  was  neoessary  to  devise  standard  methods  for 
applying  the  test  instruments,  and  to  e'onstruot  scales  for  some  of  the 
nozwdiscrete  readings*  * 

The  chief  problem  involved  was  to  sample  adeqtiately  the  very  large 
amount  of  info^’nation  that  could  be  obtained  from  tho  equipment.  This 
was  accomplished  by  threo  steps t (l)  test  points  were  carefully  selected 
to  provide  a thorough  coverage  of  the  circuit;  (2)  test  inetrnments  were 
applied  between  a test  point  and  common  ground,  never  between  two  test 
points;  and  (3)  only  common  information  aampi i w procedures  which,  could 
be  repreBshted  symbolically  were  used. 

Taking  the  AC  voltage,  CC  voltage*  and  resistance  measurements  was 
a relatively  simple  operation.  Tirst,  the  eq^pment  was  pit  in  hoimal 
operating  condition  and  these  measures  vers  taken  at  all  test  pQ^ntr  to 
provide  a standeord  with  which  to  compare  the  probl^em,  X‘eadings.  T^se 


''normal”  readings  were  taken  more  than  oxice»  and  at  different  times, 
to  correct  for  scale  reading  error  and  minor  current  fluctuations. 
Then,  each  problem  in  turn  was  put  into  the  equipment,  and  measure- 
uents  were  made.  Wherever  the  fault  had  caused  a change  in  values, 
these  were  checked  very  carefully.  They  were  cotopaured  with  the  nor- 
mal values  and  then  were  taken  again  to  make  sure  that  the  dlffoiv 
ences  were  due  to  the  fault  and  not  to  measurement  error. 

The  waveforms,  the  signal  Injections,  and  the  Information  ob- 
taiiud  from  tapping  a point  with  a screwdriver  were  special  problems. 
The  waveforms  were  taken  with  a high  quality  5*^  laboratory  type  os- 
cilloscope. They  were  drawn  on  black  paper  in  white  wax  pencil.  Later 
each  drawing  was  carefully  checked  by  direct  comparison  with  the  os- 
cilloscope waveform  again.  The  sweep  frequency  and  vertical  gain  set- 
tings of  the  oscilloscope  were  recorded  as  part  of  the  waveform  Infor- 
mat ion. 

The  receiver  outputs  derived  from  the  use  of  the  signal  generator 
and  from  tapping  on  vaxious  test  points  in  the  radio  receiver  with  a 
screwdriver  were  auditory  cues  difficult  to  symbolise.  The  informa- 
tion obtained  from  tapping  was  included  because  technic iana  often  use 
a screwdriver  or  finger  to  tap  key  points,  such  as  grids,  in  tue 
audio  section  of  a radio  receiver,  as  a crude  but  effective  method 
of  signal  tracing.  The  signal  generator  was  used  in  the  way  STs 
generally  use  it;  the  signal  was  injected  at  a test  point  and  the 
audible  effects  were  noted  at  the  speaker  of  the  radio.  Since  the 
intensity  of  the  sound  in  the  speaker  produced  by  both  taxqping  and 
the  use  of  the  signal  generator  is  the  isiportant  cue,  a four-point 


scale  was  devlsi^^-to  descrilie  the  range  of  intensities.  Sach  point  on 
the.  scale  was.  labeled. with  a short  descriptive  phrase  or  wordl  (l)  ah- 
sexit*  (2)  weak  tonSf  (3)  loud  tone,  (4)  very  loud  tone.  Tapping  pro- 
duces a pepping  noise,  so  the  word  was  substituted  for  tone  to 

describe  the  intensi;ty  of  this  noise. 

This  system  has  proven  to  be  reasonably  effective,  although  it  can 
be  Improved  for  subseqjient  uses  of  the  M^TS  Test.  The  signal  genera- 
tor, in  particular,  needs,  a more  accurate  scale  which  will  indicate 
the  output  of  the  signal  generator  and  the  output  Intensity  at  the 
radio  receiver  speaker. 

Information  from  varying  front  panel,  controls  and  screwdriver  ad- 
justments represented  another  ceding  problem.  The  important  cues  from 
these  operations  generally  come  from  observation  of  the  output  while 
the,  control  is  being  varied.  Coneeq.uently,  it  wae  necessary  to  devise 
some  means  of  representing  the  variation  in  output  through  the  range  of 
the  control  variation,  and  also  to  indicate  the  position  of  the  control 
in  the  problem,  Tor  the  radio  circuit,  the  coding  system  used  to  repre- 
sent volume  changes  is  shown  in  Fig.  3»  Short  verbal  statements  were 
used  for  the  ra.dar,  circuit. 

Referring  to  Fig,  3»  * shows  the  system,  used  for  variable  resis- 
tors, such  as  gain  controls.  Line  1 represents  the  range  through  which 
the  control  could  be  turned,  from  left  to  right,  L^e  2 represents  the 
speaker  volume  at  any  particular  point  in  this  reuige,  and  the  triangle 
(3)  represents  the  relative  position  of  the  control  when  the  problem 
readings  were  taken* 

I-A,  I-B,  I-O  are  examples  of  how  this  system  was  used.  I-A  shpwo 


that  the  control  will  cau«e  no  increase  or  . decrease  in  volume. 

I- B  shows  a moderate  Increase  In  volume  when  the  control  Is  turned  to 
the  right,  and  I-O  shows  a large  Increase. 

II  shows  the  system  used  for  variable  capacitors.  Line  1 Is  the 
range  of  variability;  2 shows  the. relative  position  of  the  control 
when  the  problem  readings  were  taken»  and  3 shows  the  location  of  the. 
signal  peak  in  relation  to  the  range  of  the  control  and  its  position 
in  the  problem, 

Again.  Il-A.  and  II-B  aroj  exemples  of  how  this  system  was  used, 

II- A  shows  a detuned  tank;  the  trimmer  is  out  of  adjustment.  • II*’B 
shows  a very  small  signal  peak  at  the  control  poaition.  but  the 
only  peak  in  its  range  of  adjustment. 

This  coding  system  was  used  to  r^resent  problem  information 
only.  It  never  showed  the  effects  of  varying  the  control  in  a nor- 
mal system,  which  ia  consistent  with  the  rest  of  the  problem  informa- 
tion. 

B,  Adminjstrat ion  of  the  Test 

The  novelty  of  the  test  eituation,  and  the  relative  complexity 
of  the  behavior  it  was  designed  to  measure,  rec^uires  that  the  sub- 
ject clearly  understand  his  role  before  he  starts.  Instructions  have 
been  wru.t ten  for  the  subjects  to  reed.  They  describe  the  test  for- 
mat briefly  and  e;cplain  the  basis  of  the  problem  information,  em- 
phasizing its  empiric^  nature.  In  addition,  the  test  administrator 
demonstrates  the  ^portent  features  of  the  test:  the  use  of  the 
informtion  and  eoswer  sections,  the  cozks.  and  the  red  warning  light, 
irinally,  a wazm^up  problem  is  given  the  subject  to  make  doubly  sdre 


he  understands  the  test.  Ourrently*  eaush  prolblen  has  a twenty  misiats 
tine  limit. 

At  the  heginning  of  the  problem*  the  subject  is  given  a card  with 
a list  of  symptoms.  He  is  given  a schematic,  and  can  refer  to  the  book- 
let of  supplementary  information;  coa5>onent  values,  and  normal  plate 
voltages.  Ho  can  sample  Information  at  any  point  he  chooses  and  can  re- 
move end  replace  as  mai^  coa^onents  as  he  chooses,  if  he  is  willing  to 
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spend  thirty  seconds  of  time  for  each  component  checked.  In  other  words, 
thr  ubject  has  to  find  his  own  path  to  solution  of  the  problem.  If  he 
is  Buccessfu] , he  locates  the  cork  corresponding  to  the  faulty  coiqoon- 
snt  and  reads  under  it  “Oear  Normal,"  signifying  that  replacement  of 
this  conqponent  with  a new  one  corrects  the  malfunction  and  ends  the 
problem, 

lY.  SCORING  PARAMBTSIS 


A scoring  system  depends  upon  the  definition  of  a criterion  of  suc- 
cess and  the  relationship  to  that  criterion  of  the  behavioral  referents 
recorded  during  the  test,  which  are  assigned  numerical  valuee.  This 
fact  means  that  there  usually  are  several  alternative  ways  to  score 


Y 

Thirty  seconds  may  appear  to  be  slight  deterrent  to  random  gues- 
sing in  the  answer  section.  Actually,  the  required  sequence  of  opera- 
tions for  turning  the  light  on  and  off  tends  to  break  up  the  subject’s 
response  pattern  and  force  him  to  concentrate  on  a manipulative  activity. 
The  light  interrupts  the  subject  at  a point  where  his  subjective  need 
to  finish  the  ta^  is  greatest;  just  before  he  can  obtain  knowledge  of 
results.  Under  these  circumstances,  thirty  seconds  seems  like  five  mixv- 
.utee,  l^irthezmore , there  are^euch  a large  number,  of  ccs^onents  in  the 
answer  section  that  only  a small  fraction  could  be  checked  in  the  twenr* 
ty  minute  time  limit  per  problem,  even  if  that  .were  all  the 'subject  did. 
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O PLATE  I.  PHOTOGRAPHS  SHOWING  THE 

STRUCTURAL  FEATURES  AND  THE  USE  OF 
THE  MASTS  TEST. 


any  teati,  during  the  developaent  of  the  MASSS  Testi  this  has  been 
kept  in  mind,  and  the  haelc  response  records  from  which  several  soor> 
ing  systems  could  be  derived  are  being  obtained  in  its  preliminary - 
tryout. 

So  far  as  scoring  is  concerned,  there  are  two  general  ways  of  ; 
treating  problems  in  the  test;  as  items  or  as  subtests.  In  this  sec<«> 

tion,  these  two  alternatives  will  be  discussed,  and  the  preliminary 
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scoring  objectives  will  be  listed. 

^ Problems  as  Items  ' 

Failures  in  electronic  equipment  are  a heterogeneous  class  of 
problem  situations.  The  dietiziguishing  characteristics  of  this  class 
are!  (a)  the  oi:dput  of  an  electronic  system  deteriorates  below  tol- 
erance limits;  (b)  this  divergence  is  reflected  in  output  symptoms 
whioh  may  be  necessary  but  seldom' are  sufficient  information  to 
locate  the  cause;  and  (c)  finding  the  cause  of  the  divergence  usually 
requires  an  examination  of  the  interacting  functions  in  tho  system 
betweeh  input  and  output. 

Heterogeneity  results  from  the  differences  among  trouble  shoot- 
ing problems  with  respect  to  number,  variety,  and  d is crlm inability 
of  output  symptoms;  number,  variety  and  discriminability  of  cues  to 
be  found  between  input  and  output;  the  specificity  of  the  relatioz^ 
ship  between  this  information  and  probable  causes:  and  the  kinds  of 
events  which  are  cauees  of  the  trouble.  These  differences  make  some 
problems  very  difficult  to  solve,  sdme  easy.  There  are,  of  course, 
other  factors  which  influence  difficulty  level,  such  as  complexity  of 


the  circuit. 


Ideall3r«  it  should  ho  posslhlo  to  sarediot  the  difficulty  lev’ol  bf 
a prohlom  from  its  discrimiziahle  eloctronio  oharactoriaticsf  and  a tost 
os^haBiziog  either  speed  or  power  could  he  constructed*  However » at  the 
time  the  prohleme  for  the  MASTS  Teat  were  select ed«  the  knowled^  neces- 
sary for  such  predictions  was  xsoking.  The  relationship  between  what 
might  he  called  the  psychological  characteristics  of  a problem  and  its 
electronic  characteristics  was  not  known.  Certain  logical  generalizar> 
tions  are  possihle*  hut  there  obviously  are  so  ocLny  determinants  of 
problem  difficulty  that  this  kind  of  generalizing  hscomss  dangerous  iu»- 
less  checked  empirically. 

The  selected  problems  were  pretested  on  a small  sample  of  subjects 
who  had  been  2STsi  and  the  results  were  used  as  a rough  indicator  of  the 
difficulty  of  each  problem.  It  is  expected  that  data  from  the  prelim- 
inary administration  of  the  test  at  the  Long  Beach  Baval  Shipyard  will 
provide  more  information  about  the  relationship  between  these  two  cat»* 
gories  of  problem  characteristics. 

When  trouble  shooting  problems  are  treated  as  i^eme  in  a test»  es- 
sentially what  is  Boored  is  the  eM  product  of  the  subject's  perfomance. 
He  solves  a problem  or  he  fails  to  solve  it  under  the  conditions  speci- 
fied by  the  test  instructions.  This  fact  can  be  zxoted  and  he  can  be 
■cored  Eight  or  Wrong  on  that  particular  yxoblein.  Whether  0 and  1«  or 
some  other  weight  is  used,  the  eseential  fact  is  that  this  procedure 
emphasizes  results  rather  than  technique.  It  cox^orms  tp  a practical 
criterion  of  a good  trouble  shooter  as  a man  who  fii4e|t,he,  cause  ef  an 
eq;alpme^;  malfunction,  -....fiivi'V 
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B«  ProblesiB  as  Sub  tests 

, ^ . , ■ . iJ. f,  ■ -I-,,  .■  * , ■ - 

Vhile  a practical I working  definition  of  good  trouble  shooting 
emphasises  results,  obviously  there  must  be  reliable  differences 
between  the  behavior  of  technicians  who  consistently  fall  and  those 
who  consistently  succeed.  We  met  assume  the  end  is  a product  of  the 
means.  In  twenty  minutes,  a subject's  behavior  should  reveal  sane 
valid  information  about  his  skill  in  trouble  shooting.  The  problem 
is  to  identify  and  score  the  elements  in  this  behavior  tidiich  discrim- 
inate among  subjects  with  respect  to  this  skill.  Unfortunately*  most 
of  the  presumably  critical  steps  in  problem  solving  are  not  directly 
observable.  Evaluation  of  them  necessarily  is  Indirect.  IHirthexmore, 
the  nature  of  trouble  shooting  behavior,  as  it  is  conceived  of  here, 
precludes  the  use  of  a checklist  type  of  procedure  which  assumes 
there  is  one  best  sequence  of  pre-solution  steps,  or  one  best  pre- 
solution path. 

Excluding  the  traditional  evaluative  techniques  which  enploy  an 
observer  to  watch  and  to  make  subjective  Judgments  about  a perform- 
ance, there  appear  to  be  two  alternatives  to  scoring  vlthin-problems 
variance.  Either  each  problem  can  be  divided  into  subgoals  or  some 
method  of  evaluating  the  subject's  technique  can  be  found. 

She  first  alternative  essentially  makes  several  items  of  each 
problem.  Eor  example,  a subject  could  be  given  credit  for  localizing 
the  fault  to  the  correct  stage,  and  additional  credit  for  localizing 
it  to  a specific  component.  The  second  alternative  is  based  on  the 
assumption  that  thqre  are  good  and 

Some  preliminary  ahaljrses  have  indicated  the  potentialities  for  de- 
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the  oireuit  in  conjunction  with  the  reeponee  reooziiet  it  is  possible  to 
reconstruct  the  subject's  path  from  beginning  of  the  problem  to  its  end. 
Path  plots  of  seireral  successf\il  and  Unsuccessful  subject's  responsee  on 
the  same  problem  have  indicated  that  the  latter  tend  to  repeat  measure- 
ments  &t  the  same  point  more  frequentlyi  may  not  definitely  localize  a 
fault  to  a stage  before  working  extensively  within  a stagei  and  tend  to 
get  into  cul-de-sacs  from  which  they  cannot  extricate  themselves,  These 
trends,  noted  from  a few  records,  will  need  support  from  a more  extensive 
analysis  before  they  can  be  useful  as  scoring  discrlminanda. 

If  - . t . 

Another  preliminary  analysis  of  the  response  records  has  been  along 
the  same  lines.  While  the  subject's  solution  path  is  being  plotted, 
judgments  of  fruitfulness  and  uxifruitfulness  were  applied  to  each  re- 
sponse. The  totals  were  used  for  cosqputlng  the  ratio  between  the  total 
number  of  responses  a subject  made  and  the  number  of  responses  which 
were  fruitful  in  moving  him  closer  to  the  solution.  This  analysis  sug- 
gested that  such  a ratio  could  be  a valid  pre-solution  measure  of  trouble 
shoctizig  skill.  It  eiophasizes  the  efficiency  of  the  subject's  technique. 
Unfortunately,  its  computation  is  somewhat  laborious,  and  the  judgments 
^ which  the  responses  are  categorized  require  more  information  for 
their  basis  than  presently  is  available* 

It  is  anticipated  that  these  and  olmllar  euialysga  of  the  response 
records  can  be  dene  more  extensively  after  the  first  administration  of 
the  MASTS  Test  has  been  completed. 


£’  3Pycli»inary  Scoring  Objectives 

During  the  initial,  administration  of  the  test  at  Long  Beach,  the 
fqllo^r^  perfuxiiuuxce  record#  were  obtained  frouL  each  subjects  (a)  time 


r«(iuir9d  to  eolve  each  probiem*  (b)  ntuabar  of  problems  solved,  within 
the  twenty  minutes  per  problem  limit*  (c)  complete  responie  records 

e 

of  pre^solution  behavior. 

The  fewest  i^sslble  restrictions  were  to  be  imposed  on  the 
subject's  behavior*  other  than  that  inherent  in  the  trouble  shoot- 
ing situation,  fheoretioally*  every  problem  selected  could  be 
solved  by  every  subject  if  he  persisted  long  enou|^h.  If  nothing 
^IsOf  he  night  rebuild  the  oircuit  a cei^onent  at  a time. 

' ^Bowsver*  for  practiced  adminietratlve  reasons*  it  was  noces- 
sary  to  .set  a time  limit  on^each  problem.  Twenty  minutes  was  se- 
lected because  pretesting  suggested  that  the  distribution  of  solution 


times  is  bimodal}  either  a subjeot  solves  a problem'  in  a relatively 
straightforward  fashion  or  he  becomes  confused  and  uses  up  a pro- 
portionately much  larger  amount  of  time  testing  the  wrong  I^otheseo 
or  fumbling  for  cues  n^ioh  will  put  him  on  the  way  to  a solution. 

The  twenty  minute  limit  was  chosen  to  cut  off  the  second  mode  of  the 
distribution. 

It  is  expected  that  the  throe  categories  of  data  will  permit 
use  of  simple  and  objective  scoring  procedures  which  treat  prob- 
lems as  items*  and  also  will  provide  the  foundation  for  deriving  3po- 
tent  tally  more  discriminating  scoring  systems*  based  on  problems  as 
subtests*  - 


See  the  append^  fjo^  a samipie  response  record* 
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Profioienoy  measuroB  should  predict  oxL-the»jo>  performance  or  some 
aspect  of  it.  !I^re  may  he. many  elements  In  the  performance  Itself 
vhi^  are  ncuv-disoriminatlngi  so  far  as  ranhing  the  performers  is  con** 
cemed,.  At  any  one  time,  the  on-th»-Joh  situation  may  not  he  r^re* 
sentative  of  the  tasks  a technic Ian. has  to  perform  over  a long  period 
of  timsc  A proficiency  measure  ahstrciots  what  are  believed  to  he 
critical  .elements  froip  the  on»the-Joh  situation*  Performance  tests 
simulate  this  situation  or  elements  of  it.  . Paper>and->penoil  tests 
usually  measure  knowledge  related  to  the  performance,  although  it  often 
is  not  A aufficient  condition  for  the  suoceeeful  ooourrenoe  of  that  petw 
formonco. 

Conventional  Joh-eample  performance  .teats  and  paper-and~penoil 
tests  may  he  placed  at  different  levels  on  a dimeneion  of  ahstraotlon 
from  the  on-the-Joh  situation.  She  MASTS  Test  was  developed  to  he  at 
a level  of  ahetractlon  intermediate  between  these  two  conventional 
'formats.  Ideally,  it  should  possess  much  of  the  administrative  convene* 
lenca  and  economy  of  the  paper-andr-pencil  type  of  test,  and  evoke  per- 
formance from  the  subjects  comparable  to  that  elicited  by  a conventional 
Job-sample  perform^ce  test. 

In  this  section,  the  MASTS  Test  is  compared  with  the  Job-eai^ple 
test  from  i^ich  it  was  derived, 

' ib*  Behavioral  Stambturi^  ih  the  T^uble  Shooting  Situation 

" The  eesentiaXs  of  this  situation  have  been  characterised  in  Sec^ 


tlon  IX.  It  iB  coneelvad  of  ae  presentlag  the  technician  with  a type 
of  complex  problem  to  eolve.  On  the  Job,  he  Is  free  to  use  certain 
methods  and  to  proceed  In  certain  ways.  He  Is  net  free  to  use  some 
other  methods  or  to  proceed  In  some  other  ways.  These  conditions 
strvicture  his  behavior;  they  Is^ose  restrictions  and  limitations  on 
it.  !?he' resulting  performance  is  a product  of  the  interaction  of  the 
individual  with  the  situation.  The  major  structuring  conditions  In 
the  on-the-Job  trouble  chootlng  sltTiatlon  may  be  listed  asl  (l)  the 
design  characteristics  of  the  equipment;  (S)  the  type  of  test  Inetrur* 
E^nts  and  supplementary  information  available;  (3)  the  regulations 
concerning  corrective  maintenance  imposed  on  the  ST;  and  (4)  the  at- 
titudes of  others  In  the  situation  toward  the  ST.  These  factors  will 
influence  the  availability  and  usefulness  of  output  symptoms»  the 
iiuTormatlon  getting  pxucedures  the  technician  uses  and  the  choice  of 
points  at  which  xo  apply  them,  the  degree  to  which  he  becomes  ego- 
involved  in  the  tasTc,  the  length  of  time  he  will  work  at  it,  etc. 

The  point  being  enphaslsed  here  is  that  his  performance  is  In' 
financed  by  the  total  situation  and  that  any  proficiency  measure  must 
inevitably  leave  out  some  of  the  behavioral  structuring  conditions 
in  the  on-the-Job  situation. 

A 4 o A ^ ^ ^ 4.  A ^ «.  A M M ^ ^ — — 4 j*“. *r? 
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of  the  MASTS  Test.  It  utilises  the  same  problems,  test  instruments, 
instructions,  scorlzig  "and  response  record  system.  The  equipment  used 
the  te^  is  tKe  Phllco  Training  Set  (Plate  XI)  , The  design  char- 


ihls  device  differ  froia  fleet  equipment  in  several 
"rejects.'  Tile  Phllco  gew  has  interchangeable  cnassis  which  corre- 
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epond  to  oirouit  stages.  Th/a  eosq^nents  in  these  chassis  generally 
are  rery  acoasslble  and  easily  identified  from  the.  schematic.  Vith 
the  exception  of  large  transformers  and  some  potentiometers,  all  the 
components  are  mounted  on  removahle  strips.  Xhis  feature  tends  to 
reduce  the  effect  of  inaccess ihility  of  components  and  difficulty  of 
their  removal  on  trouble  shooting  behavior.  It  also  reduces  the  time 
the  technician  otherwise  would  have  to  spend  in  circuit  tracing  and 
in  hunting  for  a particularly  obscure  component, 

She  circuits  in  the  Philoo  gear  are  considerably  simpler  than 
most  of  those  in  fleet  gear.  Ihle  results  in  a smaller  total  funo» 
tional.  organisation  to  voxk  with.  Shore  are  no  physical  units  larger 
than  a stage  or  a power  supply. 

Sheas  differences  are  being  pointed  out  to  make  it  clear  that 
the  development  of  the  MdSSS  Seat  has  been  a methodological  problem 
up  to  new.  It  is  e3qpected  to  resemble  most  closely  this  joh^sample 
test,  based  on  Phllco  gear,  not  any  specific  trouble  shooting  situation 
in  the  fleet.  If  the  format  of  the  test  proves  worthwhile,  its  spe- 
cific subject  matter  can  be  extended  to  fleet  equipiaent  where  neces- 
sary, Per  th«  initial  developmental  worfc,  the  simpler,  more  avail" 
able  equipment  was  chosen  in  the  interests  of  ecoiiomy  and  portability. 
However,  it  should  be  emphasized  that  the  Phllco  Job-sample  test 
still  requires  the  subject  to  perform  the  essential  trouble  shooting 
steps  discussed  in  Section  II,  and  summarized  heres  (a)  choose  inr* 
formation-getting  devices  and  procedures,  (b)  select  points  in  the 
circuit  at  which  to  sample  information,  (o)  use  infor|eat  ion-get  ting 
devices  and  procedures  correctly,  (d)  perform  the  discrimination  of 


PLATE  2.  PHOTOGRAPHS  OF  THE  PHILCO  ELECTRONICS  TRAINING  SET. 
A.  SUPERHETERODYNE  RADIO  RECEIVER. 


SIMPLIFIED  RADAR  SYSTEM. 


%sual*  or  ^Timsual"  for  sach  Tslt  of  information  collected,  (e)  inte- 
grate the  information  into  a cause-effect  frame  of  reference,  (f)  elim- 
inate posBihl.e  causes  and  check  prohahle  caiises  of  the  malfunction. 

Behavioral  itructuring  in  the  MASTS  Teat 
The  absence  of  equipment  and  teat  instruments  is  the  chief  source 
of  differences  between  the  general  structuring  conditions  of  this  test, 
and  those  of  the  Job-sample  test  discussed  above.  Adequate  representar- 
tion  of  the  problems  and  problem  information  required  a coding  system 
and  a standard  procedure  for  obtaining  the  information  to  be  coded. 

This  symbolic  representation  of  the  problems  had  its  own  implications 
for  channeling  the  subject’s  behavior?  (a)  it  eliminated  the  variance 
Inherent  in  actual  manipulation  of  test  instruments;  setting  them  on 
the  proper  scale,  connecting  them  to  the  circuit  correctly,  and  read- 
ing their  meters  correctly;  (b)  it  somewhat  restricted  the  kind  and 
amount  of  problem  inforration  available  to  the  subject  (e,g, , no  read?- 
ings  between  test  points);  (c)  it  made  getting  information  much  easier, 
since  lifting  a cork  is  simpler  than  using  test  instrpments;  (d)  it 
eliminated  the  problem  cues  ordinarily  available  to  the  unaided 
senses;  (e)  it  eliminated  the  necessity  to  relate  a schematic  to  the 
physical  arrangement  of  components  in  the  equipment;  and  (f)  it  elim- 
inated the  danger  of  electric  shock  tc  the  subject. 

Inasmuch  as  response  records  have  been  taken  during  administra- 
tion of  both  the  MASTS  Test  and  the  Job-sample  test,  it  will  be  pos- 
sible to  study  the  effects  of  the  different  structuring  conditions  in 
the  two  test  situations  on  subject  performance  and  on  problem  dif- 
ficulty, The  response  records  ere  e:Q5ected  to  be  as  important  a ^ 


Bouroe.  of  Infoncatioa  a1>out  these  effects  as  will  he  intercorrelations 
of  scores  from  tho.  two  tests.  She  results  of  these  comparisons  will  he 
presented  in  a Buhseq.uent  report. 

VI.  APPENDIX 
BESFONSS  SECOKD 

!The  sas^le  performanoe  record  reproduced  helow  was  recorded  while 
a subject  solved  a radio  receiver  problem  in  the  MASTS  Test.  The 
fiBoilt  consisted  of  a shorted  by-pass  capacitor  in  the  AVC  lead,  diode 
detector  stage.  The  subject  was  an  ST  - 2nd. 

The  left  side  of  the  sample  contains  the  reeponse  record  as  it  was 
originally  written  in  code.  The  right  side  contains  the  translation  of 
the  code.  The  column  heading  "Step"  refers  to  the  order  of  the  subject’ 
responses.  '•’Time"  was  recorded  to  the  nearest  minute,  ^location" 
refers  to  the  test  point  of  component  ircVolved  in  the  subject’s  response 
The  column  headed  "Action”  contains  the  code  numbers  for  the  different 
kinds  of  information  a subject  gets  from  the  test.  The  word  "Action” 
is  used  because  the  Information  is  categorized  by  the  information- 
sampling  devices  and  processes  which  produced  it. 


in 

iike 

' ILboatlon 

"X«ftion 

fraHfll&tion 

1 

0538 

39 

12 

laj«et«d  signal  at  plats  of  tubs 
in  first  Bide  of  audio  power  aan 
plifisr* 

2 

3« 

1 

12 

Injeetsd  signal  at  plats  of  tu’bs 
in  ssoojpd  side  of  the  audio  power 
aaplif  isr. 

3 

38 

73 

12 

Zfijjaoted  signal  at  grid  of  tube  in 
first  sido  of  audio  power  amplifier, 

4 

38 

58 

12 

» 

In^eotsd  signal  at  grid  of  tubs  in 
second  side  of  audio  power  amplifies 

R 

38 

, 74 

12 

Injected  signal  at  plate  of  phase 
splitter  tube. 

6 

39- 

68 

12 

Injected  signal  at  grid  of  phase 

- 

’■J 

splitter  tube. 

7 

39 

81. 

12 

Injected  signal  at  plate  of  audio 
voltage  amplifier  tube. 

s 

39 

85 

12 

Injected  signal  at  grid  of  audio 
voltage  amplifier  tuhe. 

9 

39 

32 

12 

Injected  elgnal  at  plates- of  diode 
detector  tube. 

10 

16 

12 

Injected  signal  at  cathodes  of 
diode  detector  tube. 

11 

18 

15 

Haasured  SC  volts  at  output  of 
diode  detector. 

12 

32 

l4 

Measured  AC  volts  at  plates  of 
diode  detector  tube. 

13 

32 

15 

Measured  3X/  volts  at  plates  of 
diode  detector  tube. 

14 

72 

l4 

Measured  AO  volts  at  plate  of  W 
as^lifier  t\fl>e. 

Keipoiite  B«oord*  («6ntlao«d) 


Step 

Time 

.Xiooation 

Action 

Translation 

05.4i 

72 

15 

Measured  W at  plate  of  .IT 

as^llf  ier 

16 

4i 

52 

l4 

Measured  AD  volts  at  screen  grid 
of  . XT  ai^llfiar  tube. 

17 

•4i 

16 

13 

Measured  ohne  (SC  resistance; 
cathodes  of  diode  detector  tdbe. 

;w’ 

IS 

13 

Mealed  ohms  (l^  reeletance)  at 
coupling  capacitor  hstween  diode 
detector  and  audio  voltage  aa^li:- 
fier. 

.19 

4i 

76 

13 

Measured  ohms  (SC  resistaunce)  on 
the  other  side  cf  same  coupling 

eapaoltor. 

20 

42 

45 

13 

Measured  ohms  (SC  resistance)  at 
A70  lead  input,  diode  detector. 

21 

4e 

0-23  1 

S— s 

Bembved  bT-pass  capacitor  (G  - 23) 
8Lnd  replaced  with  a new  oompoxw 
ont.'  (This  solved  the  problem.) 

Hote*— The  order  of  .the  .eta£ee»  frou  input  to  output » In  thle 
particular  circuit  lei  BF  aaoplifler,  KF  converter,  IP 
amplifier,  diode  detector  and  AVO,  audio  voltage  ampli~ 
fier,  phase  split ter^  and  audio  power  amplifier. 
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